04. Demo 1: Time-Based Cross-Validation

Cd13652 C6 L4 Demo 1 V2

Time Series Cross-Validation Essentials

Time series cross-validation is crucial for analyzing datasets with a temporal order, commonly used in trading and financial contexts.

  • Purpose: Helps evaluate models where data must maintain its time-based sequence.
  • Data Used: Demonstrated using Dow Jones Industrial Average index data.

Key Steps:

  1. Data Preparation:

    • Load dataset with opening and closing prices.
    • Clean data by removing unwanted characters (e.g., extra commas).

  2. Data Splitting:

    • Constraint dataset to make it divisible by intended splits (e.g., 2,500 entries).
    • Use "TimeSeriesSplit" from the sklearn library, specifying the number of splits.

  3. Cross-Validation Structure:

    • Visualization of data splits:
      • Training sets in light blue.
      • Validation sets in orange.
    • Expandable window approach: training data expands over each fold, akin to backtesting.

  4. Outcome: Through visualizations, observe how time series cross-validation configures training and validation sets, ensuring model evaluation respects the temporal data order. Useful for forecasting tasks in time-sensitive fields.

Cd13652 C6 L4 Demo 1b V2

Understanding Time Series Cross-Validation

Time series cross-validation helps evaluate models with data having a temporal nature by dividing it into sections for training and validation. Here's a breakdown of its configuration tools:

  • Gap Parameter:

    • Defines the interval between training and test data.
    • Useful for predicting targets further out in time, e.g., 30-day gaps.

  • Max Train Size:

    • Fixed size for the training set.
    • Allows moving forward through data with a sliding window approach.

  • Number of Splits (n_splits):

    • Determines how the dataset is segmented.
    • Test set size adjusts unless specified, providing flexibility.

  • Fixed Test Size:

    • Ensures the test set maintains a constant size.
    • Expanding window option adjusts the training set size while keeping test size stable.

Combining these parameters allows creating tailored configurations. Effectively assimilates cross-validation into model evaluation similarly to backtesting. Useful for forecasting or prediction tasks on time-dependent data. Suitable for assessing models while maintaining data sequence integrity.

Next Concept